Memory error prevention through static analysis and type systems

Abstract. Modern software is everywhere and much relies on it so it is important that it is secure and reliable. As humans, software developers make mistakes that may be really difficult to detect. In memory unsafe languages a large part of these mistakes are related to memory usage and management. In order to reduce the amount of bugs in software this thesis looks into using static analysis tools and other methods to automatically find where these mistakes are or alternatively preventing them altogether. This is done through a literature review. Unfortunately, static analysis results in many false positives that can take a long time for developers to sift through. For this reason many static analysis tools augment their usefulness by inserting dynamic, runtime checks in places where they are uncertain whether there is an error or not. One final approach, discussed in this thesis, for securing software memory usage, is to employ type systems or memory safe languages like Java that are designed so that the programmer is not allowed to access raw memory and make mistakes related to it. The large amount of checks that these kinds of languages must always do, result in a reduction in performance. As such all of these approaches have benefits and limitations regarding their use. The major findings were that much research has been done in static analysis tools that have managed to detect real problems. Many of the developed tools are unfortunately not available, and the ones that are available haven’t been updated in a long time or they require complicated setup reducing their usefulness

A static analysis tool for finding buffer overflows in C

Abstract. This thesis presents a new static analysis tool for C and C++, that can detect some buffer overflow errors, which are dynamic memory use related errors that happen when a program tries to read or write past the end of a memory area. The tool is implemented as a plugin for the Clang compiler in order to leverage the excellent C and C++ parsing Clang has. The new tool is ran on Clang’s abstract syntax tree (AST) representation, from which it is able to detect unsafe memory related operations that are in the analysed source code. A previous study by the author was done on static analysis theory and existing implementations in journal articles and scientific conference papers. One of the main findings was that there are no easily usable existing tools. For this reason This followup thesis set out to implement a new static analysis tool in order to start filling that found deficiency. The developed tool is available on Github at https://github.com/hhyyrylainen/smacpp. Such a tool is important in software development as static analysis can reduce the number of bugs that slip through to released versions of software. If only manual testing and automated unit testing is used on software, it leaves many problems hidden that a static analysis tool could find. This is why static analysis tools are important as not using them increases the number of problems that software developers do not find. This thesis focuses especially on dynamic memory related errors as the type of problem that static analysis is used to find. This is because many memory related issues can be remotely exploited making it a very important aspect to get right. Memory unsafe languages are ones that do not guard the programmers against incorrect dynamic memory usage, C and C++ are widely used examples of these kind of programming languages. As these languages do not guard against memory errors, static analysis is a good addition to a development workflow to catch issues before they can be exploited. The developed tool was tested with an existing test case set in order to verify that the tool can detect problems correctly in concrete programs. Because this test set contained only C programs, the developed tool focuses on them, instead of also handling C++ specific issues. In addition to the first test set another set was used for evaluating the performance of the new tool once it was completed. The new tool, when combined with Clang’s analysis as the new tool was designed to compliment Clang’s own analysis capabilities, is able to detect 4 more issues in the first test set, without adding any false positives. This means that the combination is useful. Unfortunately none of the tested tools were able to pass any test cases contained in the second test suite. The new tool also increased the number of false positives when combined with Clang, but this is likely due to many of the test cases missing the expected entry point, "main". In addition to the new tool, this thesis presents the way it was designed and how it uses Clang’s libraries to aid in the development of a static analysis tool